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During the spring 1992 term, James Monroe High School 
piloted Learning 100 On-Line, a remedial reading/literacy computer 
program developed by Educational Developmental Laboratories, Inc. The 
program, which was developed for junior high and high school students 
and adults reading at or below grade level, was implemented in two 
English-as-a-Second-Language, three Special Education, and four 
English classes. It is noted that Learning 100 On-Line is an 
integrated instruction and management system that diagnoses, 
prescribes, instructs, provides practice and reinforcement in, and 
evaluates mastery of reading, writing, vocabulary, and language 
competencies. It is noted further that it provides a mul tisensory , 
individualized approach which is designed to accoramodate a variety of 
learning styles and educational backgrounds. An evaluation of the 
impact of the program on student attendance and academic achievement 
found the pilot to be successful on several levels. Several specific 
recommendations for schools that may wish to purchase the program are 
included. This report is presented in four sections: (1) 
Introduction — program background, evaluation methodology and scope of 
the report; (2) Program Implementation — use of the system, 
perceptions of the system and its strengths and weaknesses; (3) 
Student Outcomes — reading performance and attendance; and (4) 
Conclusions and Recommendations. Two tables present summaries of 
students* pre- and posttest scores on the reading comprehension 
subtest and spring 1992 percentage of attendance by group and gender. 
(ALF) 
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EXECUTIVE SUMMARY 



During the spring 1992 term, James Monroe High Pohool 
piloted Learning 100 On-Line, a remedial reading/ literacy 
computer program developed by Educational Developmental 
Laboratories, Inc. (E.D.L.)- The program, which was developed 
specifically for junior high, and high school students and adults 
who are reading at or below grade level, was implemented in two 
English as a Second Language, three Special Educaticm, and four 
English classes. 

The pilot was successful on several levels. Teachers' 
reports of high student enthusiasm were supported by 
statistically significant gains in reading and good attendance 
achieved by students participating in the pilot. Students 
themselves agreed that working on the computer was especially 
helpful, more interesting than learning in a more traditional 
format, and fun. 

The biggest obstacle to successful program implementation 
encountered by school staff related to an inadequate amount of 
computer memory in the school's equipment, rather than to 
difficulties with a particular aspect of the Learning 100 On-Line 
system. 

Based on the findings of the evaluation, OREA makes the 
following specific recommendations to schools if administrators 
decide to purchase the program: 

• School staff should work with the superintendent's offi-:e 
to expand computer network capabilities to accommodate 
more levels of Learning 100 On-Line, so that all students 
are able to profit from using the system at the most 
appropriate level. 

• Central, district, and school staff should work together to 
insure that hardware specifications meet software program 
needs in schools purchasing the program. 

• School administrators should explore possibilities for 
making the computer lab more accessible for teachers to use 
throughout the school day, so that they can take advantage 
of the many aspects of the management system that they 
currently have little opportunity to use. 

• School administrators should facilitate meetings for the 
English, Special Education, and E.S.L. departments so that 
they may share ideas, discuss problems, and evaluate the 
program throughout the year. 
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School administrators purchasing the system are encouraged 
to provide teachers with professional development to 
facilitate efficient use of the system. 
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I. INTRODUCTION 

PROGRAM BACKGROUND AND STRUCTURE 

The Division of Instruction and Professional Development's 
(D.I.P.D.) Office of Computer Resource Services (now a part of 
D.I.P.D.) is charged with the responsibility of approving network 
managed Integrated Learning Systems (I.L^SO- As part of this 
responsibility, D^I.P^D, requires vendors who wish to sell 
I.L.S.s to New York City Public Schools to pilot their software 
in a school for one or two terms in order to assess the efficacy 
of the I.L.S. on student attendance and achievement outcomes • 
The Office of Research, Evaluation, and Assessment (OREA) is 
responsible for evaluating the computer pilots and assessed the 
impact of Learning On-Line at James Monroe High School during the 
spring 1992 term. 

Learning 100 On-Line is a remedial reading/ literacy computer 
program developed by Educational Developmental Laboratories, nc. 
(E.D.L.). The program, which was developed specifically for 
junior high and high school students and adults who are reading 
at or below grade level, was implemented in two English as a 
Second Language (E.S.L.)^ three Special Education, and four 
English classes. 

Learning 100 On-Line is an integrated instruction and 
management system which diagnoses, prescribes, instructs, 
provides practice and reinforcement in, and evaluates mastery 
of reading, writing, vocabulary, and language competencies. 
Through the use of the computer, Learning 100 On-Line provides a 
multisensory, individualized approach which is designed to 



acconunodate a variety of learning styles and educational 
backgrounds . 

The program is divided into nine reading levels ranging fro- 
first through tenth grades (Levels AA through lA) . Each level is 
divided into 20 to 30 carefully planned sequences of learning 
activities called Cycles of Instruction. 

Learning 100 On-Line is an open entry/open exit program 
which is well suited for use with students with inconsistent 
attendance patterns. It keeps track of each student's lesson 
progression, and begins instruction exactly where he/she leaves 
off. Students are not in competition with their classmates and 
are able to enjoy a sense of confidentiality using the system. 
Repetition is designed into the sequence of learning experiences. 
It is also integrated into the curriculum by using a combination 
of audio and visual stimuli to repeat every vocabulary word m 
many contexts, exercises, and learning modes. 
Diagnosis and Placemen t 

A student's entry level into Learning 100 On-Line is 
determined by a series of cloze format vocabulary tests. The 
student is placed at the lowest level at which he/she 
demonstrates nonmastery of the vocabulary. Once placement has 
been determined, the student is given criterion-referenced tests, 
so that instruction can be concentrated on the reading 
comprehension skills, taught in each cycle, in which the student 
demonstrates weakness. Nonmastery of a skill will result in a 
prescription to receive instruction in that skill, while 
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demonstrated mastery exempts the student from instruction in that 
specific skill of the cycle. 
The Management System 

The computer management system evaluates and records data on 
student progress at each step in the instructional sequence. At 
selected points this evaluation will either move the student to 
the next lesson, or "recycle" him/her through the same set of 
skills or concepts at a lower reading level, repeating concepts 
at ever decreasing reading levels until mastery of the concept is 
achieved. It then moves the student back up, having him/her 
apply this skill at gradually increasing reading levels so that 
mastery is not lost as the student moves back to his/her 
instructional reading level. 

In addition to monitoring student progress. Learning loo On- 
Line provides the teacher with an easy-to-use, menu-driven system 
designed to manage the complex delivery of instruction. It 
enables the teacher to track each student's progress from the 
teacher's computer station. It provides information on which 
questions in each lesson were answered correctly or incorrectly, 
whether the student's overall performance for that lesson was at 
a mastery level, how much total time he/she has spent on the 
computer, and his/her reading speed, when a student does not 
master a lesson, a message, which may include a prescription for 
additional instruction, is sent to the instructor. 

The management system also generates reports which provide 
information for students, teachers, and administrators including 
test analyses, prescriptions, lesson performance data, student 

history information, and class enrollment summaries. 
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To supplement the computer program, E.D.L. provides print 
material and audio cassettes for student use, 
EVALUATION METHODOLOGY 

The Division of Computer Information Services' (D.CI.S.) 
Office of Computer Resource Services* requested that the Office 
of Research, Evaluation, and Assessment (OREA) evaluate the 
efficacy of Learning 100 On-Line Pilot at James Monroe High 
School. The D.C.I.S. was interested in determining whether the 
program had a positive impact on student achievement, as measured 
by both anecdotal information about program success, and by a 
quantitative examination of students' reading improvement during 
the spring 1992 term — the term in which the pilot was in effect • 

Qualitative data included interviews with the six teachers 
administering the program at James Monroe, and with the assi^^tant 
principals of the English as a Second Language, Special 
Education, and English departments. Interviews focused on 
students' academic needs and interests, the appropriateness of 
the system for students, perceived program strengths and 
weaknesses, and suggestions for improvement. The evaluator also 
spoke with nine students participating in Learning 100 On-Line 
about what they liked and disliked about the system, and whether 
the program helped them in any way. 

OREA's quantitative evaluation of the program included 
analyses of reading test scores, attendance, and the amount of 



*This office has subsequently been transferred to the Division of 
Instruction and Professional Development. 
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time studants spent using the system ("time on system"). Pre- 
and posttest scores on the comprehension subtest of the Stanford 
Diagnostic Reading Test (3rd Edition, Green Level, Form G) were 
used to measure reading progress « Evaluators selected this test 
as the most appropriate instrument available to measure reading 
progress in the Special Education and limited English proficient 
(LEP) students who participated in the pilot •* OREA 
administered the comprehension subtest to students at the 
beginning and again at the end of the spring 1992 term, and also 
used central files to compare the spring 1992 attendance of 
students in the pilot with their attendance during the previous 
spring (spring 1991) . 

The total amount of time students spent working on the 
computer was then compared with students' scores on the reading 
comprehension subtest to determine whether there was a relation- 
ship between the amount of time students spent working on the 
system and their performance on the reading test. 
SCOPE OF THIS REPORT 

Chapter II summarizes staff and student comments about their 
use of the system, as well as their perceptions of the program's 
strengths and weakness. Analyses of reading scores and 
attendance are provided in Chapter III, while Chapter IV presents 
OREA's conclusions about and recommendations for the program. 



*One of the three Special Education classes was included in the 
pilot but did not participate in the testing. 
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II. PROGRAM IMPLEMENTATION 

USE OF THE SYSTEM 

The Components of the Learning 100 On-Line system are 
interrelated. Each lesson in the system contains a vocabulary 
(Language Clues) , comprehension (Reading Strategies and Thinking 
Strategies) , and writing (Write and Read, and Writing Strategies) 
segment. New vocabulary is introduced through an audio/ visual 
presentation. The vocabulary words are then used to teach 
reading and listening comprehension skills. These skills are in 
turn reinforced by creative writing lessons and lessons in 
grammar, mechanics, usage, and the process of writing 
compositions. At the end of a cycle students enjoy recreational 
reading (Go Books) which incorporate the vocabulary which has 
been taught and reinforced throughout the cycle. 

Two English as a Second Language classes, three Special 
Education classes, and four English classes, taught by two 
teachers from each department, participated in the Learning 100 
On-Line pilot program during the spring 1992 term. Students in 
the English and E.S.L. classes used the computer lab three times 
per week, and worked in their classrooms on other aspects of the 
curriculum during the remaining two days. 

The Special Education classes also used the lab two to three 
times per week, although they had initially used the program five 
periods weekly. Special Education teachers reduced the nnmber of 
days in the lab when it became apparent that the level of the 
system was too difficult for many students, who became 
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frustrated, one Special Education teacher opted to eliminate the 
Reading Strategies segment of the program to avoid this problem, 
and used only vocabulary lessons for students who tested below 
the level at which they were operating. This same teacher was 
especially enthusiastic about the system's writing component, 
however, noting that students' increased confidence in their 
ability to write was apparent in the increased length of their 
written work. Special Education teachers encouraged students who 
were unable to write their own stories to dictate them to the 
paraprof essional . 

The E.S.L. Department piloted Learning 100 On-Line with 
their highest level (7 and 8) students because staff considered 
the system too difficult for less advanced students. However, a 
teacher commented that, "there are endless opportunities for 
F.r:.!.. [students^ to use ^'his program] on lower levels." 
PERCEPTIONS OF THE SYSTEM 
Teachers ' Attitudes 

In general, the teaching staff expressed approval of the 
system. The English teachers and the assistant principal of the 
English department believed that the program provided students 
with good preparation for Regents Competency Tests.* The 
assistant principal for Special Education found the program to be 
"very important for learning disabled students," and expressed an 
interest in expanding the base of Special Education teachers who 

♦Regents Competency Tests (R.C.T.s) are minimum competency tests 
in several subject areas that students must pass prior to 
receiving their high school diploma. 
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are trained in the use of the system. Teachers from both the 
English and the E.S.L. departments noted, however, that two days 
per week in the classroom is not a large enough block of time in 
which to adequately develop other aspects of the curriculum. 
Conversely, they wished that they had more time to supplement the 
computer lab work in class. 
Students' Attitudes 

Teachers reported a generally high level of enthusiasm for 
the program among students who were working at their appropriate 
level. The OREA evaluator observed that most students turned on 
their computer immediately upon entering the lab, and began 
working on their lessons before the teacher came in. When asked 
what they liked about Learning 100 On-Line, several students 
mentioned the multi«sensory approach as being especially helpful. 
"You can cr^rcenVrate a I0I: . "^n^itead of using your mouth, you can 
use your fingers," one said. "It shows you more than talking. 
It makes things more interesting and fun," commented another. An 
E.S.L. student observed that "Listening and writing words helped 
me learn them. I have many friends who need help in English. 
This program is the answer." Other comments included, "I like 
computers. They're cool," "It lets you practice," "I learned how 
to understand what I am reading," and "The computer helps me out. 
It shows me what I get wrong." 

Of course, students' comments raise the question of whc^^ther 
Learning on-Line 100 is successful because of its software or 
merely because students enjoy working on a computer. Subsequent 
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evaluations of computer pilot programs should address this 
question more fully. 
PROGRAM STRENGTHS 

Teachers and administrators' comments about the strengths of 
Learning 100 On-Line centered around students' motivation and 
attitudes, praise for individual components of the system, and 
teacher support engendered as a result of the system. Staff 
agreed that students enjoyed working on the system and were 
developing positive attitudes toward computers and greater ease 
in using them. They singled out the individualized nature of the 
program (which allowed for a wide range of reading and writing 
levels) , the varied exercises (which minimized student boredom) , 
the dictionary, the management system (which enabled students to 
monitor their progress) and interesting subject matter and 
writing assignments as particular procjrani strengths. Finally, 
teachers and administrators praised the thoroughness of the staff 
training provided by E.D.L., and the establishment of a cross- 
departmental faculty support group, as important outgrowths of 
their experience with the system. 
PROGRAM WEAKNESSES 

The biggest problem that staff encountered in implementing 

the Learning 100 On-Line system was the fact that their computer 

network's memory could not accommodate all of the levels that 

students needed.* S taff initiated the pilot with levels 

*According to central staff, the additional memory was not 
invested in because of the pilot nature of the program. However, 
hardware specifications will be clear and purchased for schools 
purchasing the system in the future. 



applicable to students reading between the third and the seventh 
grade levels (Levels CA through FA) , with the understanding that 
the Bronx superintendent's office would facilitate upgrading 
their computer capabilities to allow for the addition of the 
lower levels (AA, and BA) which some of their students required. 

This upgrade was not in place by the end of the school year. 
Consequently, 35 students who tested as reading below the third 
grade level were required to work one or two levels above their 
prescribed placement. In addition, the appropriate level was not 
available for students who placed at the CA level or above, but 
who, because of nonmastery of a skill, were referred back by the 
management system to a lower level. In an interim attempt to 
address this need, the E.D.L. representative suggested that 
teachers supplement instruction at the CA level with print and 
audio cassette materials geared toward the lower levels (AA, BA) . 

In addition, teachers in the program had limited access to 
the computer room during periods when they were not teaching. 
This limited access reduced the amount of time available to the 
teachers to work with the various reports that are designed to 
assist them in the instructional process. There also appeared to 
be some difficulty in obtaining assistance when problems arose 
with the system. According to some teachers, the computer 
coordinator, who was responsible for other computer rooms, was 
not sufficiently knowledgeable about the program to be able to 
address specific problems. Teachers saw a need for someone to be 
available to provide immediate information or help. 



staff generated several suggestions to improve the program, 
E.S.L, teachers believed that the system would be improved for 
their students if an audio-lingual component, directed 
specifically at limited English proficient students, were 
included. The E.S.L. assistant principal suggested that program 
material be revised to include topics of particular interest to a 
Latin American or Haitian student population, as well as topics 
emphasizing career awareness. Other suggestions included 
lessening the system's reliance on male-oriented stories,* 
expanding the audio component throughout lessons rather than just 
at the beginning of a lesson, and providing more screen space for 
the writing component. 



*Program staff reviewed the entire Learning 100 On-Line curri- 
culum to ascertain whether there was, in fact, an overabundance 
of male-oriented stories. This was not found to be the case. 
E.D.L. staff suggested that the teachers who believed that there 
were more male-oriented stories may have been working with only a 
portion of the curriculum. 
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III. STUDENT OUTCOMES 

READING PERFORMANCE 

Complete pretest and posttest reading score information was 
available for 55 students who participated in the pilot program. 
Raw scores on the reading comprehension subtest were converted to 
normal curve equivalent scores (N.C.E.s)* by hand, using a 
conversion manual obtained from the test publisher. According to 
these data, students gained an average of 8.5 N.C.E. units from 
the pretest to the posttest. This gain was statistically 
significant (t=5.4; df=l,54; e<.01) indicating that students' 
reading comprehension was meaningfully improved from the 
beginning to the end of the pilot. 

Further analyses of test scores by gender (female/male) as 
well as by type of student (English as a Second Language, Special 
Education, General Education) were extremely positive and are 
reported in Table 1. As shown in this table, on average, the 
reading scores of students in E.S.L. classes, special education 
students, and students in English classes all improved between 3 
and 14 N.C.E. units. Such a consistent pattern of gains in 



*A normal curve equivalent (N.C.E.) score is similar to a 
percentile rank, but is based on an equal- interval scale ranging 
from 1-99, with a mean of 50 and a standard deviation of 21. 
Because N.C.E. scores are equally spaced, arithmetic and statis- 
tical calculations such as averages are meaningful, and compari- 
sons of these scores can be made across different achievement 
tests. A gain in N.C.E. units from pretest to posttest indicates 
that a student's performance is better than what would be 
expected by normal growth and chance. 
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Table 1 



Summary of Students* Pretest and Posttest Scores 
on the Reading Comprehension Subtest, 
in Normal Curve Equivalents 







Pretest 


Posttest 


Mean 


Group 


N 


Mean 


S.D. 


Mean 


S.D. 


Difference 


E.S.L. Female 


13 


10.22 


11.70 


13 . 50 


15. 63 


3.28 


E.S.L. Male 


12 


17.56 


11. 12 


24 . 03 


14. 18 


6.48 


Special Ed. 
Female 


4 


2.43 


2 . 85 


10.28 


9.32 


7.85 


Special Ed. 
Male 


5 


1. 00 


. 00 


12.64 


18 . 04 




General 
Education 
Female 


8 


15.70 


12 .08 


23 . 63 


15. 36 


7.93 


General 
Education 
Male 


13 


20.22 


13 .21 


35. 11 


15.25 


14.89 


Total 


55 


13.58 


12.45 


22 . 06 


16.88 


8.49" 



This difference was statistically significant (t=5.42; df=l.54, 



• OveraM, students' reading comprehension improved from the 
beginning to the end of the pilot. 
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students who might be expected to have some difficulty with 
reading comprehension speaks very positively for the program. 

Interestingly, a very low correlation (r=.i3) was found 
between time on the system and posttest gain. This low 
correlation suggests that there is very little relationship 
between the amount of time students spent using Learning lOO On- 
Line and their test scores. Perhaps this finding can be better 
understood in light of students' comments indicating that they 
enjoyed working on the computer, and the evaluator's observation 
that students' appeared self-motivated when using the system. It 
may be that giving students the opportunity to work on a computer 
fier se motivates students to make measurable progress in reading, 
and that this motivation is not tied to a specific amount of time 
they spend on any particular computer system. 
ATTENDANCE 

OREA also calculated the spring 1992 attendance of students 
in the pilot. Complete data were available for a total of 119 
students. According to these data, and as shown in Table 2, 
students in the pilot had a spring 1992 attendance rate of 78.0 
percent overall. E.S.L. students' attendance was highest at 87.4 
percent followed by a 75.6 percent attendance rate among General 
Education students in the pilot and a 70.1 percr^nt rate for 
Special Education students. in general, there were slight 
differences in the attendance of students by gender: on average, 
the attendance of boys was slightly higher than that of girls. 
Interestingly, the average attendance of pilot students was very 
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Table 2 



Summary of Spring 1992 Percentage Attendance 
by Group and Gender 



Group 



Female 



Male 



Combined 



E.S.L. 86.9 

Special Education 73.8 

General Education 68.0 

Total 76.4 



88.0 
68 .6 
82.1 



79.3 



87.4 
70. 1 
75.6 



78.0 



• Attendance varied slightly by group and gender. Overall, 
girls attended class slightly less often than did boys. 

• E.S.L. students had the highest attendance of any of the 
groups. 
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similar (78.0 percent) to the percentage reported for all of 
James Monroe High School (78.5 parcent) during the spring 1992 
term by the Division of High Schools on Period Attendance Reports 
(PARS) . 
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IV. CONCLUSIONS AND RECOMMENDATIONS 



The spring 1992 pilot of the Learning 100 On-Line system at 
James Monroe High School was successful on several levels. 
Teachers' reports of high student enthusiasm were supported by 
statistically significant gains in reading and good attendance 
achieved by the special education, general education, and limited 
English proficient students who participated in the pilot. 
Students themselves agreed that working on the computer was 
especially helpful, more interesting than learning in a more 
traditional format, and fun. 

The biggest obstacle to successful program implementation 
encountered by school staff related to an inadequate amount of 
computer memory in the school's equipment, rather than to 
difficulties with a particular aspect of the Learning 100 On-Line 
system. 

Based on the findings of the evaluation, OREA makes the 
following specific recommendations to schools if administrators 
decide to purchase the program: 

• School staff should work with the superintendent's office 
to expand computer network capabilities to accommodate 
more levels of Learning 100 On-Line, so that all students 
are able to profit from using the system at the most 
appropriate level. 

• Central, district, and school staff should work together to 
insure that hardware specifications meet software program 
needs in schools purchasing the program, 

• School administrators should explore possibilities for 
making the computer lab more accessible for teachers to use 
throughout the school day, so that they can take advantage 
of the many aspects of the management system that they 
currently have little opportunity to use. 
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school administrators should facilitate meetings for the 
English, Special Education, and E.S.L. departments so that 
they may share ideas, discuss problems, and evaluate the 
program throughout the year. 

School administrators purchasing the system are encouraged 
to provide teachers with professional development to 
facilitate efficient use of the system. 
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